A Random Forest Guided Tour

Tree Prediction

Each tree estimates the response at point \(x\) as: \[ m_n(x; \Theta_j, D_n) = \frac{\sum_{i \in D_n(\Theta_j)} \mathbf{1}_{X_i \in A_n(x; \Theta_j, D_n)} Y_i}{N_n(x; \Theta_j, D_n)} \] where \(D_n(\Theta_j)\) is the resampled data subset, \(A_n(x; \Theta_j, D_n)\) is the cell containing \(x\), and \(N_n(x; \Theta_j, D_n)\) is the count of points in the cell.

Forest Prediction

The forest estimate for \(M\) trees is: \[ m_{M, n}(x) = \frac{1}{M} \sum_{j=1}^{M} m_n(x; \Theta_j, D_n) \] where \(M\) is the total number of trees, \(m_n(x; \Theta_j, D_n)\) represents the prediction from each tree, and the forest average yields the final prediction.

Random Forest Regression

Splitting Criteria:

• The CART-split criterion is used to find the best cut:

• \(N_n(A)\): Number of data points in cell \(A\).
• \(Y_i\): Response variable for observation \(i\).
• \(\bar{Y}_A\): Mean of \(Y_i\) in cell \(A\).
• \(AL\) and \(AR\): Left and right child nodes after the split.

Stopping Condition:

Nodes are not split if they contain fewer than nodesize points or if all \(X_i\) in the node are identical.

Prediction:

\[ m_{M, n}(x) = \frac{1}{M} \sum_{j=1}^{M} m_n(x; \Theta_j, D_n) \]

• \(M\): Total number of trees in the forest.
• \(m_n(x; \Theta_j, D_n)\): Prediction from the \(j\)-th tree.

Random Forest Classification

Splitting Criteria:

• The Gini impurity measure is used to determine the best split:

\[ \text{Gini}(A) = 2 p_{0, n}(A) p_{1, n}(A) \]

where:

• \(p_{0, n}(A)\): Empirical probability of class 0 in cell \(A\).
• \(p_{1, n}(A)\): Empirical probability of class 1 in cell \(A\).

Prediction:

• Each tree makes a prediction using the majority class in the cell containing \(x\).
• Classification uses a majority vote:

\[ m_{M, n}(x; \Theta_1, \ldots, \Theta_M, D_n) = \begin{cases} 1 & \text{if } \frac{1}{M} \sum_{j=1}^{M} m_n(x; \Theta_j, D_n) > \frac{1}{2} \\ 0 & \text{otherwise} \end{cases} \]

where:

• \(m_n(x; \Theta_j, D_n)\): Prediction from the \(j\)-th tree.
• \(M\): Total number of trees in the forest.

Like this…

[3]

The Data

Analysis - Stutti

Analysis - Matt

Confusion Matrix

• 0: Non-high-revenue OPCO
• 1: High-revenue OPCO

Model Statistics

ROC Curve Analysis

Figure 1: ROC Curve for High Revenue Prediction

Feature Importance

Figure 2: Feature Importance for High Revenue Prediction

Analysis - Mika

Conclusion

Mika, you got this!

… coming soon